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ABSTRACT 
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participant understandings of a course evolve through the process of 
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knowing already familiar in the academic context. An attempt is made 
to demonstrate hov the model has been adapted to the unique 
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is given to the applicability of the approach to curriculum 
evaluation situations in higher education institutions. 
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y ABSTRACT 



In recent years there has been an increasing awareness of the role of 
Curriculum Evaluation in Higher Education. A wide variety of models and. 
methods for evaluation currently compete for our attention. Some of these 
models are limited in scope, focusing too narrowly on student achievement 
on tests of cognitive outcomes; some are almost entirely attitudinally-based, 
being concerned primarily to discover whether students have enjoyed or been 
engaged by the learning experience offered in a course; some require 
substantial* investments of time and thus depend upon the availability of 
specialist evaluators within higher education institutions and pose certain 
political problems about the production and distribution of evaluation 
information to those involved in a course and in the institution more generally. 
In this paper a conception of evaluation is developed which depends upon the 
self reflective activities of those most closely involved in the development 
and teaching of a course. 

The model for self reflection is based upon the critical theory of Jurgen 
Habermas. It invites participants in a course to form a self critical community 
and to examine the relationship of theory and practice in the teaching of a 
course and in its substance. The relations of theory to practice and of 
practice to theory are manifested through the organisation of a course. By 
making critical analyses of the way theory expresses itself in practice through 
organisation and of the ways practical problems are registered in organisation 
posing theoretical difficulties, it becomes possi •? for participants to gain 
a concrete and authentic insight into the nature of the course as a learning 
experience. The model presented here focuses upon the way participant 
understandings of a course evolve through the process of critical debate. 

Since universities already encourage the formation and maintenance of 
critical communities through their research function, this evaluation approach 
capitalises upon ways of knowing already familiar in the academic context. 
Moreover by working simultaneously at the three levels of theory, organisation 
and practice it avoids the first two limitations of evaluation models listed 
above, and by involving participants in a course in the self reflective 
enterprise, it overcomes the third limitation of most current approaches. 

The self reflective approach to evaluation has recently been tried in the 
Curriculum Development Centre in Canberra, where a community of Curriculum 
Developers face common problems and issues related to their work across a variety 



1 A paper prepared for the Fifth Annual Conference of the Higher Education 
Research and Development Society of Australasia^ Mou: * Cravatt College of 
Advanced Education, Brisbane, Queensland, May 11 - 1A, 1979. 



of curriculum areas. Early experience with the mo:lel shows that too rlgourous 
an adherence to its three foci (Theory, Organisation and Practice) tended to 
make self reflective discussion somewhat rigid and formal; later experience 
has shown that it can be adapted so that it becomes more problem oriented and 
better adapted to the convivality of discussion. In the paper some of these 
practical difficulties are discussed and recommendations made about the organisations 
of self reflection in critical communities. An attempt is also made to show 
how the model has been adapted to the unique conditions of the Curriculum 
Development Centre and how it may be adapted for use in wider situations. 
Particular attention is given to the applicability of the approach to 
curriculum evaluation situations in higher education institutions. 



INTRODUCTION 



In recent years, there has been increasing interest in curriculum 
evaluation in higher education, spurred on, no doubt, by the barbs of an 
accountability movement which attempts to respond to economic adversity through 
"rationalisation" of higher education provision and the promises of an emerging 
technology of evaluation that it can provide the means by which "rationalisation" 
of — if not rationality in — educational provision can be achieved. In such 
a climate, universities and colleges tike the University of Illinois have 
attempted to preempt the intensification of external evaluation by getting in 
first with a systematic procedure for University " self -evaluation" In such 
a climate, university and college higher education research units are 
increasingly called upon to carry out course evaluations and to assist 
embattled administrations in making hard choices about the "rationalisation" of 
institutional offerings. In such a climate, Paul Dressers Handbook of Academic 
Evaluation (1976) appeared, somewhat ominously subtitled "Assessing Institutional 
Effectiveness, Student Progress, and Professional Performance for Decision 
Making in Higher Education". 3 

In such a climate, the oracular, the opportunistic and the officious in 
higher education research units will find the invitation to play a larger 
technical-managerial role in the improvement, of higher education institutions 
hard to resist. 

The central claims of this paper are: 

(a) most curriculum evaluation technologies now being developed are 
sufficiently narrow in scope to be sharply limited in utility; 

in consequence they may all too easily become the political tools 
of autocratic or bureaucratic administrations; 

(b) curriculum evaluation in higher education is by no means a new 
activity, and has already developed convivial processes for course 
improvement and institutional self-regulation by critical 
communities of course developers and scholars; 



2 The University of Illinois established a Council on Program Evaluation (COPE) 
to carry out evaluations of all departments of the University on a five-year 
cycle, using a combination of internal review and external review mechanisms. 

3 Reviewed by the first author for the Journal of Curriculum Studies (1977 , 
9, 91-94). 
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(c) by the systematic development of these convivial processes in the 
method of critical self-reflection it is possible to achieve the 
justifiable ends of institutional and curricular improvement 
without underestimating the complexity of the intellectual and organ- 
isational tasks involved or endangering the quality of higher education 
by replacing academic concerns with administrative ones. 

These claims will be defended by a necessarily brief critique of current 
technologies of curriculum evaluation in higher education, by demonstrating 
that self-critical and convivial evaluation processes do indeed exist in 
contemporary curriculum practice in higher education, by outlining a perspective 

on evaluation and self-reflection which is sufficient for most evaluation 
purposes in higher education, by describing some recent experience at the 
Curriculum Development Centre, Canberra, where we tested the persepctive in 
practice, and by referring to some of the problems encountered with it and 
suggesting some directions for improvement. 

I CURRENT TECHNOLOGIES 

As the curriculum development movement of the 1960 f s spawned its progeny 
of evaluation methods and models, the developers of those models and methods 
began to apply them to their own work as teachers and curriculum developers 
in universities and colleges. While few of these higher education evaluators 
regarded measurement of student outcomes (both affective and cognitive) as 
sufficient for course evaluation, let alone faculty or institutional evaluation, 
a considerable stress was laid upon these outcomes as indicators of success 
in course teaching (Costin, Greenough and Menges, 1971). They may be 
relatively efficiently gathered, they are clearly relevant, and they sometimes 
suggest how the teaching of_a course or the course itself may be improved. The 
efficiency of such approaches to course evaluation, however, made the technology 
of measurement appealing as an evaluation mechanism; though limited, it made 
at least a genuflection in the direction of evaluation as an obligation upon 
teachers. 

The development of instruments for course evaluation demanded a new breed 
of specialists in higher education: institutional researchers whose task was 
to provide a technology for course evaluation. Two consequences followed: 
first, evaluation came to be identified with the technology of instruments, 
and second, as the institutional researchers recognised the complexity of the 
evaluation task it became their responsibility to develop the more complex 
technologies necessary to satisfy the requ5 r - ~nts of an expanding role. 

With the benefit of hindsight, it is ' to see that outcomes-oriented 
approaches are severely limited. Where tlK, .ocus on student attainment and 
student attitudes*, they ignore many issues of considerable importance in — 
reaching an evaluation of a course (e.g. substantive concerns, teacher 
satisfaction, resource requirements, the quality of the learning milieu). 

Even more important than their limitedness, however, is the question of 
the relationships between teachers, students, administrators and evaluators 
that these arrangements created and sustained. Inevitably, they raised questions 
about control in the production and distribution of evaluative information, 
and questions about the uses to which such information was to be put (e.g. in 
decisions about promotion, course improvement, or resource allocation) . The 
independence of the evaluators could not be guaranteed by the "objectivity" 
of their methods; it also required the development of procedures for handling 
data which could stand outside the information-flows already serving decisions 
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about promotion, course improvement and resource allocation. Who were 
the evaluations for? The students, the teachers, or the administrators? 
Or perhaps all? These questions were not easily resolved, and they remain 
thorny ones. (See, for example, MacDonald, 1976, and House, 1976) 

What is at stake here is that the emergence of a specialism of 
institutional researchers and evaluators has created a new battery of technical- 
political issues in higher education evaluation. In some senses, this makes 
the issues more tractable because they are explicit. Nevertheless, we will 
argue that self-reflective processes in evaluation are, for most purposes, 
superior to evaluation approaches which depend upon "external" evaluators. 

It is a mistake to think that the technology of outcomes evaluation 
was the only approach taken by higher education evaluators. Indeed, some of 
the most interesting developments of the "illuminative" approach ( p arlett and 
Hamilton, 1976) have been worked out in higher education settings. (See, for 
example, Miller and Parlett f s Up to the Mark> a study of the examination game 
in higher education) • But these developments also call for specialist 
evaluators: "illuminative" approaches are time-consuming and dependent on the 
availability of experienced evaluation f ieldworkers . Not every course, 
department or institution can release the resources necessary for such 
evaluation studies. 



II "INFORMAL" SELF-CRITICAL APPROACHES 



In any course, department or institution, there is always a low but 
significant level of evaluative activity underway, even when it is not supported 
by "formal" evaluation studies. This activity is the more or less systematic, 
more or less public form of self-reflection that accompanies the design and 
teaching of any course, or the operation of any administrative unit in an 
institution. This "substrate" of evaluative activity feeds the usual social 
negotiation processes of course committees and institutional management. It 
depends upon the self-critical awareness of participants in the institution. 

Such evaluative activities are convivial, being based on "natural" processes 
of social negotiation, deliberation, justification and decision making. And 
they are inherently political, affecting the conditions of social life of 
those within and outside the institution. They are essentially practical, being 
concerned primarily with real courses or real administrative units within the 
institution, but they are necessarily guided by matters of principle and of 
theory as these are canalised into the organisational forms of the institution 
(for teaching, research, public service and administration) . 

The v problems with these informal mechanisms for evaluation are that they 
are often insufficiently systematic (being based on the resolution of practical 
problems that actually do arise), insufficiently comprehensive (being guided 
by moment-to-moment deliberation and decision making rather than by systematic 
consideration and cross-referencing between matters of theory, organisation and 
practice), insufficiently rigorous (being based on the vagaries of discussion 
rather than explicit analysis and systematic observation), and open to abuse 
(often being unreasonably influenced by individual opinion or political 
expediency). Nevertheless, such mechanisms are adaptive and evolutionary: 
decisions taken are open to correction in the light of experience and new forms 
of organisation of the institution's work can be created to accomplish new 
tasks. The difficulty is that they are sometimes slow to respond: beachheads 
of power may be difficult to disperse once they become institutionalised, 
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suggestions for change based upon student response to a particular course 
may be hard to accommodate within forms of organisation which exist across 
the whole institution, and long-term consequences (e.g. the success of 
graduates in their careers) may be ignored by the day-to-day self-regulatory 
processes in the institution. 

The theoretical problem for evaluation is whether these informal processes 
constitute a defensible form of evaluation. Then, assuming that they are, 
the organisational problem for evaluators is how to improve them. . 

It is our view that these informal mechanisms do constitute a defensible 
form of evaluation. * The characteristic that defines them as such is their 
self-critical quality. They certainly conform to our own definition of 
evaluation as the process of marshalling information and arguments which enables 
interested individuals and groups to participate more fully and more effectively 
in the critical debate about & program. 

The organisational problem is to find ways that these informal processes 
can be intensified to overcome the practical problems listed earlier, while 
preserving as much as possible of their practicality (and relevance) and their 
conviviality. In the next section, we develop a perspective on evaluation 
as self-reflection which may satisfy these requirements. 

Ill EVALUATION AS SELF-REFLECTION IN A CRITICAL COMMUNITY 



The approach to evaluation we advocate here attempts, first of all, to 
overcome the limitedness and political manipulability of many current approaches 
to curriculum evaluation. Secondly, it attempts to overcome the inadequacies 
of informal self-reflection listed in the last section. Thirdly, it attempts 
to capitalise on the existence and experience of self-critical groups in higher 
education insticutions so that evaluation can be more rigorous and yet remain 
under participant control.^ 



Theory and Practice 

Many evaluation approaches are theoretic in character (e.g. the tests and 
measurements approach, Lindvall and Cox, 1970; the "goal-free 11 approach, Scriven, 
1974; or the "objectives" approach, Tyler, 1949) . That is to say, they proceed 
from theoretical aspirations (either in subject-matter or in terms of the 
market justifiability of a program) to create an evaluation mechanism for 
determining shortfalls in a program or its performance. Other approaches are 
essentially practical (e.g. Stake f s "responsive" evaluation, 1975). That is to 
say, they proceed from the conduct of a program and the perspectives of those 
in and around it to create an evaluation mechanism which can identify issues 
and agreements about the program as a justifiable organisation. If we take 
the aims of evaluation to be the justification of programs and their improvement 
towards more justifiable forms of organisation, then we inevitably raise questions 
about the relations between theory and practice in a program. 



4 The approach presented here is based on the critical social science of Jurgen 
Habermas (1972,1974). It was developed in a doctoral thesis by the first author 
(Kenanis, 1976) and worked out in procedural terms during a consultancy to the 
.Curriculum Development Centre, Canberra in 1978 (Kemmis, 1978). It was 
implemented as a joint project By both authors at the Curriculum Development Centre, 
and continues to be used in modified form by the second author. 



ERLC 



7 



The process of evaluation, whether theory- or practice-driven, consists 
of explicating the relations between theory and practice to show how program 
performance relates to aspiration, how aspirations are justified (e.g. in 
evaluation of program goals), and how program practice is constrained by the 
opportunities and the circumstances of the setting in which the program is 
tried. 

There are precursors to this idea. For example, Stake's (1967) 
"countenance" matrices are especially thorough-gbing in relating descriptive 
and judgmental data about a program; its antecedents, transactions and outcomes; 
and logical and empirical consistencies between these elements. The "countenance" 
approach is not ordered explicitly around the relations between theory and 
practice expressed in, a program (in aspiration and performance), but these 
relations are strongly implicit in the approach. Few approaches to evaluation 
are as comprehensive. 

Evaluation models which are theoretic in character suffer from the 
4; " incompetences of theory: limitations of scope, abstractness, and idealism. 
Models based on practice suffer from the incompetences of the practical: the 
problems of reducing multiplex reality to communicable ideas, vulnerability to 
circumstanc s and expediency. Only by developing a model which can contain 
these contraries is it possible to avoid oversimplification from the perspective 
of principle (e,g. evaluation as a description of shortfalls between performance 
and theoretical ideals) or from the perspective of practice (e.g. rewriting 
program goals so that they can be attained under given circumstances) . The 
present approach does this by projecting the relations of theory to practice 
into the plane of "organisation", which is in the nature of plans for action 
guided by principles; i.e., organisation is considered as a concrete and 
anactable expression of principles which mediates between theory and practice. 

Moreover, most evaluation models are guided by a discrete or "enclosed" 
conception of the evaluation task. It is presumed to finish with a decision 
or a judgment. They are thus limited by present aims or ideals and by present 
conditions of operation. Theoretic evaluation approaches often attempt to 
overcome this difficulty by concerns for generalisation (beyond the conditions 
of present practice); practical approaches by offering alternative perspectives 
on the program from which justifications or rationalisations may be synthesised. 
The present approach attempts to overcome these limitations by organising 
itself around the evaluation process (self-reflection and debate), and by 
allowing for provisionality at every level (theory, organisation and practice). 
It takes an evolutionary stance, seeing the program as adjusting itself to 
present constraints at each level, yet attempting to become more coherent 
within each level and between levels. But the c rive towarlds coherence is not 
merely towards self-justification narrowly cone -ved (in its own terms, or for 
limited conditions); it is countered by concerns forced upon it from outside 
its own development (the wider contexts of theoretical debate, organisational 
arrangements in society, and practical circumstances). In short, the program 
is never isolated in its justification of its work, and justification is never 
a once-and-for-all matter. The present approach begins from the assumptions 
that programs and their justifications are time-dependent and evolve through 
time, and that they are contextually-embedded. 

Higher education institutions already encourage the formation of critical 
communities through their research and academic teaching functions. And in every 
field, understanding evolves through the systematic testing of relations 
between theory and practice (e.g. in experimentation, observation, the systematic 
collection of historical records, or testing of resilience of ideas by argument). 
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In every field, organisations exist for public examination of these theory- 
practice relations (publication of research reports, professional conferences, 
etc*)* Curriculum evaluation in higher education, by encouraging the 
development of critical communities around the curriculum functions ot the 
institutions, can capitalise on the familiarity of staff with these processes 
and create the conditions for reflexive development of the work. (Such 
processes are surely assumed in any argument for academic freedom.) 



The self-reflective approach 

In curriculum development and evaluation, theory, organisation and 
practice are always interrelated* A curriculum is guided by theoretical 
principles, expressed in organisations for teaching and learning, and manifested 
in practice in real settings. Different principles apply in considering a 
curriculum at each level. (These are summarised in an Appendix to this paper.) 
Theory develops by research and scientific discourse; organisation by planning 
and the flexible implementation of plans in the light of circumstance; practice 
develops in learning by doing and from mistakes. 

Organisations for curriculum express theoretical and value commitments 
about social action, and are modified both in the light of developments in theory 
and in the light of practical problems in implementation or changes in circumstance. 
Theory develops partly according to its own internal principles, but also by 
being expressed in organisation and tested in practice. Practice is guided 
by theoretical principles and organisational constraints, but the point of 
theory and of organisation is that they may control the purely circumstantial 
process of trial and error. 

Furthermore, each level has its own distinctive "real-world 11 context which 
influencfs it, and these influences flow on to the other levels. Theoretical 
ideas develop through discourse in' critical communi .ies of researchers; 
organisations interact with other organisations and within wider social, 
administrative and legal frameworks; practice is constrained by local conditions. 
Through £uch mechanisms, contextual concerns "seep in" to a curriculum-in-action. 

In considering these as part of a comprehensive evaluation enterprise, it 
is necessary to opetate with different principles at each level* But within 
the evolutionary perspective of the approach, it is possible to say something 
about the three levels in general . Problems in programs arise as problems of 
inflexibility or intransigence on the one hand, and as problems of the unbridled 
proliferation of alternatives on the other. The first set of problems arise 
because or ganisr.tion, theory or practice remain unchanged in the light of 
changed circumstances; the second set of problems arise because of a drift towards 
incoherence, organisational instability, or uncoordinated action. 

In evolutionary terms, organisations adapt themselves at every level by 
the gen/ ration of variants and processes of selection among variants. The 
aspiration to greater awareness creates new or variant ways of thinking, 
organising and acting; the aspiration to coherence or self-regulatory control 
creates a tendency to suppress new ways of thinking, organising or acting. The 
self-reflective process consists in negotiating between these tendencies in 
the service of both increased awareness and increased control or coherence. This 
is achieved differently at: each level. 

Organisation is in the nature of plans. It is expressed in strategies, 
blue-prints for action, routine management and control operations, working 
relationships expressed as roles and links between roles, and the like. 
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Since organisation is about plans, we may think of it by analogy with law 
which evolves both at the parliamentary level and by precedent in common law 
by a more or less self-regulatory process. Taking the parliamentary aspect of 
the analogy, we may say that organisation evolves by legislation and amendment. 
New prescriptions for and proscriptions against action are enacted as laws which 
provide guidelines for action. In the light of experience, they may be modified 
by amendment. Legislation "organises" action, and new legislation appears as a 
phenomenon of the generation of variant forms of organisation. Experience 
exerts a selective function on these variants, and leads to amendments. In 
this sense, legislation and amendment is a systematic evolutionary process. 

Theory is about ideas. It is expressed in theoretical propositions, 
statements of curriculum rationales, statements of individuals' understandings, 
aspirations, values and beliefs, and the like. 

Popper (1974) discusses the evolution of 
scientific knowledge in the language of "conjecture and refutation". Toulmin 
(1972) gives a rather less formalistic treatment of the evolution of knowledge, 
and one which overcomes some of the problems of seeing knowledge as evolving 
purely formally; by contrast, Toulmin demonstrates the power of critical 
communities in the evolution of knowledge. Nevertheless, Popper's phrase is 
apt, and will be retained here. New or variant ideas are conjectured by 
researchers who attempt to refute or disconfinn them through research 
programmes (whether experimental, as in science; interpretative, as in history; 
or analytic, as in philosophy). Conjecture £.nd refutation thus appear as 
twin variation and selection processes in Uhe evolution of theory. 

Practice is about action. It is evident in the work itself, expressed in 
the activities and forms of life of participants 4 n the work. 

At the level of practical action, variation apd selection appear in the 
process of trial and error. Systematically pursued (i.e., by acting in -a 
spirit of experimentation), trial and error can become a systematic evolutionary 
process. 

In a "living" curriculum, these processes naturally interrelate. Self- 
reflection, as an evaluation process, is the systematic attempt to interrelate 
them. 

This whole set of relationships might be expressed in the following schema: 



Insert Figure 1 here 



Figure 1 expresses the general outline of relationships which must be 
considered in critical self-reflection. The task of the evaluation enterprise 
is to identify these relationships, to discover problems within and between levels 
and to stimulate the generation .of potential solutions. 

Within the schema, it can be seen that theory modifies organisation by the 
influence of conjectural principles on legislated forms of organisation; 
organisation modifies theory as amendments to organisation suggest refutations 
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of conjectured principles. Organisation modifies practice as legislated forms 
of action prescribe and procribe trials in action; practice modifies organisation 
as errors or failures of action suggest amendments to legislated forms of 
organisation. 

The links between theory and practice are indirect; as has been suggested, 
they are mediated by organisation. (Theory and practice "speak different 
languages", as it were; organisation, whether in research or social action, 
mediates between them, making them "comprehensible" to one another). Nevertheless, 
it is as well to note the apparent relations between them: theory modifies practice 
through the apparent relation of conjectured principles to trials in action; 
practice modifies theory through the apparent relation of errors or failures of 
action to the refutation of conjectured principles. 5 



An example i 

In carrying out an evaluation according to the principles outlined here, 
it becomes immediately apparent that an enormous intellectual and analytic 
ti^sk is imposed on any critical community bold enough to embark upon systematic 
sfilf-reflection. What is at stake here is not the ultimate impossibility of 
tha task — it engulfs huge realms of human knowledge and social principle — 
but the ultimate indefensibility of any alternative. In short, if evaluations 
must always be limited, then they should be limited and open rather than 
limited and closedT The approach denies evaluation models which would make any 
absolute or transcendental claims about the justifiability of programs, and 
evaluations which claim that all justifications must be tailored to prerent 
circumstances (rationalisations). It raises questions to be considered, 
recognises their contextual and historical embeddedness, and urges thos^ 
involved in the reflective process to go further in their analyses and observations. 
Most of all, it urges those involved in a program to set out their principles, 
their plans, and their practice so that they are available for reflection. 

Imagine a university biology course. Just to raise some of the issues 
involved in the approach, consider the kinds of theories upon which it depends: 
t>eories of knowledge of the particular subject-matter (e.g. evolutionary theory), 
of curriculum and pedagogy (teaching and learning, etc.), cad social theory 
(ethics, ideology, political economy). These are all expressed in the 
organisation of the course: its processes of justification and the structure 
of arguments presented in the course; the organisation of the subject-matter-to- 
be-presented (in reading, lectures, practical work, etc. — the specific contents 
of the course); the arrangements made for teaching and learning and the use of 
supporting resources (e.g. lectures, practical work, tutorials, library work, 
assignments and examinations); and the organisation of teacher-student, teacher- 
teacher, student-student and other social relationships (concerning, for example, 
notions of teachers 1 authority, students 1 rights, the role of administrators, 
and the like) . And these in turn may be related to specific forms of activity 
in practice: the conduct of argument and justification in the course; the 
specific contents actually studied; the nature of lectures, practical sessions, 
and the other component parts of the course; and the specific social relations 
which come to exist between specific individuals. 



5 A range of other considerations should be considered against the background 
of this framework. Some are touched upon in the summary of principles appended. 
Still others, like the social theory they imply, must be discussed in a future 
paper • 
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Figure 2 mentions these relationships without further explication, but 
could raadily be expanded on the principles expressed in Figure 1. 



Insert Figure 2 about here 



The second main claim of this paper is that reflection of this kind already 
goes on in higher education institutions as courses are informally evaluated 
by those who participate in them in one way or another, and that self-reflective 
meetings already exist where such matters are discussed. The third claim of 
the paper is that these discussions could be intensified and improved by 
making these meetings more systematic: by explicating these kinds of relationships 
observing more closely, by recording, and by acting in a spirit of experimentation 

The task cannot be comprehensively performed given the real constraints on 
time, social context and participation in a course. Using the approach, the 
schema, and the principles it employs, however, it may be possible to extend 
the range of issues to be discussed in evaluating a particular program within 
the limits of time available, the social context of the course, and the people 
most directly affected by it. In short, the approach demonstrates what thorough 
justification would be like and invites interested parties to participate in 
relevant discussions without closing them off through a technology of evaluation. 
It is, if you like, a plea for reasonableness — for "rationality as reasunablenes 
to use Weir's (1976) term — in the face of contemporary climate for evaluation 
based on rationalisation (in both its economic and its intellectual senses). 

IV SELF REFLECTION MEETINGS IN THE CURRICULUM DEVELOPMENT CENTRE (CDC) 

Ba ckground information on CDC 

The Curriculum Development Centre was established in 1975 to develop school 
curricula and school educational materials at the national level. In order to 
perform this Tunction the Act establishing the Centre empowered CDC to 

undertake research 
publish materials 

collect, assess and disr tvJ natc information. 

The professional staff of the Centre reflects these functions and consists 
of people with training and experience in educational research and evaluation, 
teaching, selected curriculum subject areas, information services (library work 
and computerised dat*: banking), administration and educational publishing. Thus 
there is a considerable diversity of professional abilities and functions which 
have to be brought together in various combinations to work on curriculum 
projects. 

Developing common understanding of the work of the Centre, its underlying 
philosophy and concepts, methods of working (particularly the ways of working 
cooperatively with. the education systems and schools) and the practical problems 
which delineate and constrain programs, is essential to effective team work 
within the Centre. 
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The organisational arrangements for any one project are generalised 
Figure 3. As is evident from Figure 3, the structure provides the opportunity 
for numerous meetings and widespread discussion of any aspect of a project* 



Insert Figure 3 about here 



Theory 

The proposal that CDC adopt the method of self-reflection for its internal 
evaluation was put forward at a general staff meeting at the end of June, 1978. 
The central thrust of the proposal was that CDC should view itself as a 
organisation that learns from experience and gradually builds up increasing 
understanding of and competence in curriculum development processes. As a aid 
to deliberate fostering of the collective learning of CDC, as opposed to the 
individual learning that inevitably takes place on the job, self-reflective 
meetings were to be held. The discussions in the meetings would use the 
concepts of theory, organisation and practice and the interaction between these 
as the basis for analysis and assessment. The idea was to hold a series of 
meetings In CDC'fe program areas (the areas where developments are underway) and 
to trace the development of programs over a period of time. At intervals, wider 
staff meetings would be convened to look at the main findings from individual 
discussion groups and thus foster the development of the Centre staff as a 
organisation that learns. 

If that was the theory behind the self-reflection program, what of its 
organisation and practice! 



Organisation 

The organisation was essentially simple. Three people with a background 
in evaluation (Ed Davis, Clare Hughes and Stephen Keranis) would offer to convene 
meetings on programs and projects at the request of other CDC staff. One of 
/these people would act as chairman and in the initial experimental meetings all 
of us would participate to gain experience in the process. The chairman would 
start the meeting by describing the use* of the concepts of theory, organisation 
and practice as the basis for analysis. 

In order to keep work to a minimum, record-keeping was to be quite informal. 
Each member of the group was asked to note down points which he considered 
important. All the notes were collected and filed as raw data - no writing 
or typing up* Meetings could also be recorded and the tapes kept for reference. 
For future meetings, one member of the group would look at the file and summarise 
key issues as seen by the group at the time. In that way it was hoped that on- 
going view of the development of the project could be maintained. 



Practice 

What actually happened? 

Ten meetings have now been held on the following topics: 
Language Development Project, 
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Curriculum Information Project, 

Expressive Arts Project, 

Small-scale grants, 

Mathematics education, 

Bib lio graph ies , 

Dissemination, 

Library services, 

Publications, 

Core Curriculum program. 



The list appears to display an immediate weakness - no program has been 
discussed more than once. However, this should not be taken as indicative of 
failure. One reason for the lack of on-going meetings about programs is 
that events move slowly and evaluation of ideas and changes in organisation and 
practice take time and reappraisal is not needed frequently. The second reason 
is the wide scope for discussion of projects generally. In some cases, steps 
have been taken partly as a result of the self-reflection meetings to set up 
structures within projects to continue this function; in others, existing 
structures have been better used. The accumulation of experience is taking 
place through the involvement of staff in discussion of a number of projects 
rather than through on-going analysis of one program. Continuing analysis of 
individual programs and recording of program evolution may come in time. 

The first three meetings were held on an experimental basis. At that 
point, about half of the members of staff had been involved in self-reflective 
meetings and continuation of the meetings was offered as service to the Centre 
by the Studies and Inquiries section. Despite favourable comments on the initial 
meetings there was a total lack of response to the invitation — a not altogether 
surprising outcome in an organisation which has had a record of being unable to 
sustain any system of staff meetings, formal or informal. 

In the light of favourable comment from participants in the initial meetings, 
we were reluctant to see a potentially good idea lapse. We therefore looked for 
ItiBVMB which seemed to be of importance to CDC and the second author convened 
two more meetings, one on small scale grants and one on bibliographies. These 
two meetings went particularly well; partly because all the people at the 
meetings were involved in small scale grants activities and in compiling and 
using bibliographies (though in different programs) and had not discussed the 
topics before, and partly because it was possible for the second author, who 
chaired the meetings, to analyse fairly clearly at the start of the meetings 
what the concepts of theory, organisation and practice might mean in those areas. 

The last three meetings have been at the request of other members of staff, 
which represents something of a breakthrough. It seems that the discussions 
are being used to some extent as preliminary problem solving devices in that 
staff are seeking to clarify their ideas and obtain the views of others in a 
non-hierarchical situation before moving into the more formal decision-making 
structures of the Centre. Beyond that, the discussions are serving a felt need 
to exchange ideas on issues and processes common to a number of projects. 

While generally the focus has been those concerns which are under the 
control of Centre staff, discussions have ranged into the wider areas of the 
function of CDC as a national organisation, the setting of priorities for the 
Centre and CDC decision-making processes. 
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The theory of self-reflection in an evolving organisation as expounded 
earlier in this paper seems too grand and too ambitious, to describe the 
relatively modest activities that have taken place in CDC, Yet it still seems 
essentially sound* Ideas have evolved and affected the organisation and 
practice of the Centre. Understanding of what we are about has been enhanced. 
The most effective meetings have been those in which specific aspects of 
theory, organisation and practice have been analysed or in which one level 
has been selected out for particular attention. Practice has shown, however, 
that there is a need for one person, or a small group of people, to be responsible 
for the organisation of the program of self-reflective meetings. This person 
must generate meetings (and thus has to be in a position to identify topics 
of general concern and interest) and respond to requests from members of staff. 
The program needs to be provided with impetus, and to some extent to be driven, 
or in a busy organisation it will lapse. 



V REFLECTIONS 

Expedience with the self^reflective approach suggests that too rigorous an 
adherence to its analytic categories (organisation, theory and practice) makes 
it cumbersome. If analyses at each level were too detailed, discussion became 
rigid, formal and self-conscious. In order to retain the conviviality of 
discussion, it was necessary to relax its analytical strictures and allow 
discussion to rove rather more loosely around the three foci. Discussion also 
seemed to be improved when the approach was introduced in such a way as to 
encourage explorations of ideas at each level — these could be refined as 
experience with the approach accumulates. 

Discussions also seemed to be improved when they began from organisation 
and led from there to discussions of practice and theoretical principles. On 
the whole, identifying theoretical principles and analysing theoretical questions 
remains the most difficult analytical task. 

It is clear from the meetings held, however, that they do not provoke the 
defensive reactions usually associated with evaluation: these were disc ussions, 
often stimulating and insightful ones, between communities of individuals who 
were concerned with the improvement of programs of common interest and with 
their justification. But because the conversations were among constructively 
critical colleagues, they were often penetrating. 

A major problem is that the meetings have not yet generated well-articulated 
justifications across the analytic categories. It is perhaps too early to 
expect that. But the records actually collected, while they are suggestive, are 
by no means adequate. The approach begins by asserting the value of discussion 
and debate; at worst, it may end in a talking-shop. But here, too, there is 
room at least for hope: participants have felt better prepared after self- 
reflective meetings to justify their programs publicly (e.g. to meetings of 
the CDC Council), and there is some evidence that they have influenced project or 
program documents. This is encouraging, because the approach is predicated on 
the idea that Justification of a program is not a once and-for-all matter, and 
that it develops along with the program itself. 

Higher education institutions already engage in self-reflective activities 
of the kind envisaged by the approach. It is unlikely that course-based 
self-reflection would be very much more rigorous than was the case for CDC. 
It might be possible, however, to improve the rigour of discussion at the 
theoretical level -- indeed, higher educationists are often acutely awaro of 
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theoretical issues (as, for example, in debates over students' rights, the 
structure of knowledge in disciplines, and the proper pedagogy in courses). 
Whether these concerns can be interrelated, or whether they can be related to 
organisation and purely practical considerations remains to be seen. 

On the one hand, constraints of time and interest in self-reflection seem 
to suggest that the approach, as outlined here, is too comprehensive and too 
analytically-detailed to be practical; on the other, it is clear that the 
theory and organisation of self-reflection will need to be still further 
explicated. It is our hope that it can be modified in the light of its own 
principles, and more effective forms for the organisation of self-reflection 
discovered. It s perspective offers a coherent approach to analysis of 
theoretical issues at stake in a given curriculum, and a way of generating and 
synthesising observations of its practice. It may thus contribute to the 
process of critical debate about programs among participants in the communities 
of interest they create. 
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APPENDIX : SUMMARY OF PRINCIPLES FOR SELF-REFLECTION 



A. The principles in general 

Aim : Improvement: living with greater awareness and greater 

self-control. (Assumptions of autonomy and responsibility 
of individuals, free commitment to joint action by fair 
negotiation between participants and consensus about the 
negotiation processes.) 

Problem-Sources: Intransigence/inflexibility (stability in the face of 
changed circumstances); unbridled proliferation of 
alternatives in relatively stable or benign circumstances. 

The dialectic; acting in a spirit of experimentation by 
thinking critically about real problems (concrete 
negation); generating solutions within the limits of 
individual ability, the capacity of the organisation, 
and the limits of understanding, and trying them out 
with an eye to their implications within and across levels. 

Critical awareness and democratic control. 

The work of the institution conceptualised as an evolving org- 
anisation which takes seriously its commitments in organising 
work, developing sound theory/rationale and developing 
sensible practice. 

Democratic society, critical communities, communities 
of self-interest. 



* Method: 



Criteria: 
Content: 

Con text: 



B. Organisation 

Aim: Improvement of organisation by making it more authentic 

and more appropriate. 

Problem-Sources: Intransigence of present forms of organisation; inflexibility; 

proliferation of new ways of working when procedures are 
already available; problems of diminished sense of 
participants' control. 

Method : Legislation and amendment; forward planning and flexible 

implementation in the light of principles and circumstances. 
The method is concretely developed in : 

(a) description of present organisational structures, 

(b) identification of problems in organisation itself 
and in relation to theory/rationale and practice, and 

(c) generation of potential solutions and testing them 
within organisation and in relation to theory/ 
rationale and practice (acting in a spirit of 
experimentation) • 

Criteria: Authentic insights in the organisation of enlightenment, 

authentic consensus about the organisation of the 1 work. 
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Content: 



Context: 



Strategies, work-plans, blueprints for action, routine 
management and control operations, working relationships 
and structural arrangements (e.g. in organisational 
charts), etc. 

Content of institutions, laws, regulatory procedures. 



Theory /r a t iona le 



Aim: 



Prob lem-Sour ces : 



Method: 



Criterion: 



Content: 



Context: 



Improvement of theory/rationale through increased internal 
and external consistency; increased intellectual control- 
over the work. 

Intransigence of present theory/rationale; blind commitment 
of theories, values; proliferation of theoretical statements 
or commonsense understandings without systematisation; 
incoherence; romanticism; abstraction; irrelevance. 



Conjecture and refutation; argument and research, 
method is coacretely developed in: 



The 



(a) 
(b) 



(c) 



formulation and expression of theory/rationale, 
formulation and consideration of issues through 
analysis, research and relationships with organisation 
and practice, and 

generation of potential solutions to problems testing 
them within theory/rationale (internal consistency 
and in relation to organisation and practice (external 
consistency) • 



True statements (justified true belief reached through 
scientific discourse). 

Theoretical propositions; statements of rationale; 
statements of participants' understandings, aspirations and 
values* 

Context of theory, scientific discourses, debate, 
justification and negation among critical communities. 



Practice 



Aim: 



Problem-Sources : 



Improvement of practice through better adaptation to 
working conditions and contexts of action; adaptation to 
theory/rationale, and organisation; acting with greater 
wisdom and prudence; adaptation to the purposes of 
participants and achieving free commitment to the work. 

Intransigence and conservatism of present practice; 
undisciplined proliferation of alternative ways of 
working (without regard for the coherence of the work); 
tendencies to act arbitrarily, purely opportunistically, 
or purely reactively. 
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Method: Trial and error; learning by doing and learning from 

mistakes. The method is concretely developed in: 

(a) recounting and reviewing activities, 

(b) identification of problems, failures, mistakes, 
both within practice and in relation to theory/ 
rationale and organisation, and 

(c) generation of potential solutions and testing 
them in practice (internal coherence) and in 
relation to organisation and theory/rationale 
(external coherence) . 

Criterion:' Prudent decisions. 

Content: Action: the work itself; forms of life and activities o 

participants. 

Context: Circumstances and opportunities presented by particular 

situations. 
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CONTEXT OF ORGANISATION 



ORGANISATION 



LEGISLATION 



< > 



AMENDMENT 





CONTEXT OF TIORY 



CONTEXT OF PRACTICAL ACTION 



Figure^: Interrelationships of Organisation, Theory and Practical Action 
in an Evolutionary framework, 



ORGANISATION 



1 Processes of justification, structure of 
arguments in a course. 

2 Reading, lectures, prac-work, etc;content 
e.g. predator-prey relations. 

3 Course structure; arrangements for teaching 
and learning, e.g. lectures and tutorials 
and assignments and examinations. 

k Teacher-student relations, etc (e.g. 
students' rights, teachers' authority), 




1 Theory of knowledge, e.g. evolutionary 
epistemology. 

2 Theory of subject-matter, e.g. 
evolutionary theory. 

3 Theories of curriculum and pedagogy ' 
(teaching and learning) 

4 Social theory: ethics, ideology, 
political econony. 



The rhetorical and justificatory form of 
a particular lecture, a practical session, 
a tutorial, an experiment, etc. 
Specific contents studied, e,g, simulation 
of predator-prey relations in a hypotheti- 
cal population. 

Specific topic and organisation — e.g. the 
lecture on statistical models of predator- 
prey relations. 

Specific social situation, e.g. this lecturer 
lecturing to these students in this situat- 
ion. 



Figure 2 : Some interrelations between organisation, theory and practice in an 
imaginary biology course. 
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Specific to the Project 



For CDC as a whole 



Outside CDC 
organisation 



Institution in which project 
is located 



(For externally 
funded project) 



E 



Managemen t/ Steering/ Advisory 
Committee for Project. 

(Project Officer plus 
2-5 others) 




CDC 

■Resource 
staff 
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Project 
staff 



CDC Internal 
Management 
Committee 
(CDC staff) 



(I ; or Projects 
operating in 
States). 



State Project Officers 



n 



Task forces on curriculum 
development teams 




Minister (Federal 
Department of 
Education) 



Senior Management 
Committee (Senior CDC 
staff) 



Field Services Meetings 

(Representatives of State, 
Catholic and Independent 
Schools who act as contact 
officers and meet twice a 
year) 



Study Groups 

(Short term group convened 
to advise CDC on priorities 
for research and development 
in a particular area) 



Education Systems 
(State, Catholic 
and Independent) 
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Figure 3 : Organisational Structure for a project, 
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